Frame completions for optimally robust reconstruction 

Matthew Fickus'^, Dustin G. Mixon and Miriam J. Poteef^ 

"Department of Mathematics and Statistics, Air Force Institute of Technology 

Wright-Patterson Air Force Base, Ohio 45433, USA 

^Program in Applied and Computational Mathematics, Princeton University 

Princeton, New Jersey 08544, USA 

-^ ■ ABSTRACT 

Cn ' In information fusion, one is often confronted with the fohowing problem: given a preexisting set of measurements 

about an unknown quantity, what new measurements should one collect in order to accomplish a given fusion 
task with optimal accuracy and efficiency. We illustrate just how difficult this problem can become by considering 
one of its more simple forms: when the unknown quantity is a vector in a Hilbert space, the task itself is vector 
reconstruction, and the measurements are linear functionals, that is, inner products of the unknown vector with 
given measurement vectors. Such reconstruction problems are the subject of frame theory. Here, we can measure 
the quality of a given frame by the average reconstruction error induced by noisy measurements; the mean square 
■^H , error is known to be the trace of the inverse of the frame operator. We discuss preliminary results which help 

|Jh ' indicate how to add new vectors to a given frame in order to reduce this mean square error as much as possible. 
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1. INTRODUCTION 

The synthesis operator of a sequence of vectors F = {fn}n=i i'^i ^^^ is F '■ IR^ ^^ M.^' , Fy := X]n=i y('^)/'i- That 
^ , is, _F is the M x N matrix whose columns arc the /n's. Here and throughout, we make no notational distinction 

^SJ ' between the vectors themselves and the synthesis operator they induce. The vectors F are said to be a frame 

for R^' if there exists frame bounds < A < B < co such that A\\x\\^ < ||F*xp < B\\x\\^ for all x e R^^. In 
CTS ' this finite-dimensional setting, the optimal frame bounds A and B of an arbitrary F are the least and greatest 

eigenvalues of the frame operator: 

\^ . N 

O; FF*^Y.fnfn, 

n=l 

respectively. Here, /* is the linear functional /* ; M^^ — ;• M, /*/ := (/,/„). In particular, we have that F is a 
. ^_, ^ frame if and only if the /„'s span M^^, which necessitates M < N. 

r> I Frames provide numerically stable methods for finding overcomplete decompositions of vectors, and as such 

j^ ■ are useful tools in various signal processing applications. '' "' Indeed, if i^ is a frame, then any x € M^^ can be 

decomposed as: 

N 

x^FF*x^J2'^x,U)fr,, 

n=l 

where F = {fn}n=i is a dual frame of F, meaning it satisfies FF* = I. The most often- used dual frame is the 
canonical dual, namely the pseudoinverse F = {FF*)~^F. Note that computing a canonical dual involves the 
inversion of the frame operator. As such, when designing a frame for a given application, it is important to retain 
control over the spectrum {Xm}m=i of FF* . Here and throughout, such spectra are arranged in nonincrcasing 
order, with the optimal frame bounds A and B being Am and Ai, respectively. 

The mean square error is one way to measure the quality of a given dual frame F. In particular, consider 
the problem of reconstructing x from F*x -\~ e, where e is an additive error term. Applying any given dual F to 
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F*x + e gives F{F*x + e) = FF*x + Fe = x + Fe. The magnitude of the error between the original vector x 
and its reconstructed approximation x + Fe is ||i^e||- Treating e as a random variable, the corresponding mean 
square error (MSE) is the expectation of ||-Fe|P with respect to e's probability density function p: 



MSE[F] ;= / ||Fe|pp(e)de. 



In the special case where F is the canonical dual {FF*)~^F and the entries of e are independently distributed 
with each having mean zero and variance ct^, the MSE can be simplified ' in terms of the trace of the inverse of 
the frame operator: 

M 

MSE[{FF*)-^F] = a2Tr[(FF*)-i] = ^2 ^ --. (1) 

771 — 1 

In order to construct reconstruction systems that are robust to noise, we try to design frames for which the MSE 
(1) is as small as possible. One (nonrcalistic) way to do this is to simply scale one's frame vectors F. Indeed 
for any c > 0, the frame operator c^FF* of cF has eigenvalues {c^ ^m}m=i ^^^ ^o the corresponding MSE tends 
to zero as c becomes large. However, this method for lessening MSE is unrealistic in real-world communications 
applications, since there the magnitudes {|(a;, fn)\}n=i oi the entries of the transmitted signal F*x are bounded 
in terms of the signal-to-noise ratio of the given channel. As such, the problem of minimizing the MSE is usually 
considered in the context of frames F in which the nth frame element /„ is required to have a given prescribed 
length |[/„||2 = fin- 

In particular, choosing ^„ = 1 for all n = 1, . . . , TV, one may consider the problem of minimizing the MSE (1) 
over all M x N matrices F with unit norm columns. This problem has been solved, ' the answer being that such 
an F is necessarily a tight frame, meaning its lower and upper frame bounds A and B can be taken to be equal, 
namely FF* = Al for some A > 0. Such unit norm tight frames (UNTFs) have the property that their N x N 
Gram matrices F*F have ones along their diagonal, and so their M x M frame operators FF* = Al satisfy: 

MA = Tr(AI) = Tr(i^F*) = Tr(i^*F) = N. 

Thus, the tight frame constant A of a UNTF is necessarily A = ^. As such, the MSE of a UNTF F is: 

)ri=l '" )ri=l 

Note that due to the -^ term in (2), for any fixed M and a this MSE will tend to zero as N grows large. That 
is, for a given signal x and a fixed signal-to-noise-per-transmitted-coefficient parameter ct, one may expect to 
communicate x with arbitrarily high levels of reliability even through a noisy channel, provided one is willing to 
first encode x via an A'^ x M matrix F* of a correspondingly high level of redundancy. That is, this reliability is 
purchased at the cost of computing and transmitting F*x, which is a longer signal than x itself. 

We are interested in generalizing these ideas to the realm of information fusion. There, we assume that we 
have already been given F'^ x where Fjq^^ = {fn}n=i i^ ^ fixed known M x Nq matrix. That is, we assume that 
we are given a set of inner products {{x, fn)}nli- If ^JVo is a frame, then x can be reconstructed from these 
measurements. However, this reconstruction may lack stability if F^Vq is poorly conditioned; in such cases the 
MSE of the canonical dual of F^^ is large. We therefore seek to add measurement vectors to this frame — to 
complete F^g = {/n}„=i to a longer matrix F/v = {fn}n=i where A^ > A'o — in a manner so that the MSE of the 
canonical dual of Fn is minimal. Here, we again formulate the problem realistically by prescribing the norms of 
these new measurements. To be precise, we are interested in solving the following problem: 

Problem 1. Given vectors F^o = {fn}n=i in ^^^ , an integer A^ > A^o and a sequence of desired norms 
{f^n}n=No+i' ^^^ vectors {fn}n=No+i i^ ^^^ *^^^ havc thc property that the mean square error Tr[(F^F^)^^] 
of the resulting completed frame Fn = {fn}n=i is minimal. 

In the remainder of this paper, we discuss some of our recent progress towards solving this problem. 



2. CONSTRUCTING FRAMES WITH EIGENSTEPS 

Solving Problem 1 will require us to have a good understanding of how the spectrum of a frame operator 
can change as a result of the inclusion of new frame elements of known prescribed norms. We have recently 
made progress^'" in solving a less-difficult version of this problem: Given nonnegative nonincreasing sequences 
{Xm}m=i ^nd {/i„}^=i, construct all frames F = {fn}n=i ^i^h the property that FF* has spectrum {Xm}m=i 
and that |i/n|P = M" for ^'U "-• This work is based on the classical notion of eigenvalue interlacing. 

To be precise, we say that a given nonnegative sequence {(3m}m=i interlaces on another such sequence 
{am}m=\, denoted {amj'^Ji C {/3m}m=i, provided ^,„+i < a.m < Pm for ah m = 1, . . . ,71 - 1. This notion 
of interlacing can be extended to sequences of identical length: we say that {am}m=i != {/^m}m=i provided 
otM < Pm and /3m+i < cum < Pm for all m = 1, . . . , M — 1. It is classically known that if G is self-adjoint, then 
the spectrum of G + //* interlaces on that of G; given a sequence of vectors F = {/ri}^=i, we apply this fact to 
the frame operators of the partial sequences F^ = {fm}m=i'- 

n 

F F* = ^ f f* 

Letting {Xn-m}m=i denote the spectrum of i^„F^, we have that F^_^_iF*_^_-^ = F^F*+f^^^J^^^ and so {Xn+i-ra}m=i 
interlaces on {Xn-^m]m=i- Such a sequence of interlacing spectra is known '^'^ as a sequence of (outer) eigensteps: 

Definition 2. Let {A„i}*f^]^ and {/inj^^i be nonnegative and nonincreasing. A corresponding sequence of outer 
eigensteps is a sequence of sequences {{Xn;m}m=i}n=a which satisfies the following four properties: 

(i) Xo-m ~ for every m = 1, . . . , M, 

(ii) Xn-tu = Xrn for every to = 1, . . . , M, 

(iii) {A„_i;„J^f^i C {Xn-,n}^^-^ for every n = l,...,N, 

(iv) Em=i ^r,;m = Em=i Z^™ ^1" ^vcry n = l,...,N. 

The following result, namely Theorem 2 of Cahill et aVs work,' characterizes the existence of a sequence of 
vectors whose frame operator possesses a given desired spectrum and whose elements have given desired lengths: 

Theorem 3. For any nonnegative nonincreasing sequences {Xm}m=i ^'^'^ {l^n}n=i! ^very sequence of vectors 
F ~ {In]n=i *'^ ^^^ whose frame operator FF* has spectrum {X,n}m=i '^'^'^ which satisfies ||/n|P = IJ-n for all 
n can be constructed by the following process: 

A. Pick outer eigensteps {{Xn-m}m-i}n=a '^■^ i''^ Definition 2. 

B. For each n — 1, . . . , N , consider the polynomial: 

M 



Pn{x) := J^ (a:^- A„;„). 



Take any /i G M.'^^ such that H/ip = fii- 

For each n ~ 1, . . . , N — 1, choose any fn+i such that: 

\\Pn:xfn+lf = - 1™ (x - A)^^^±iM VA G {Xn;,n}fLl- (3) 

x^X Pn[X) 

Here, Pn-\ denotes the orthogonal projection operator onto the eigenspace N(AI — F^^F*) of the frame 
operator of Fn :~ {fni}m=i- ^^^ limit in (3) necessarily exists and is nonpositive. 

Conversely, any F constructed by this process has {A„i}^=i o,^ ^^^ spectrum of FF* and ||/n||^ — A'n for all n, 
and moreover, F^-^F* has spectrum {Xnim}m=i- 

The method of Theorem 3 is the first known algorithm for producing all such frames. However, there are two 
issues with this method with respect to implementation. Wc discuss the first issue in Section 3 and the second 
issue in Section 4. 



3. CHARACTERIZING EIGENSTEPS 

Our first issue with the algorithm of Theorem 3 is that Step A is vague. Indeed, for a given {Xm}m=i ^^^ 
{l^n} n=i, it is not clear whether or not a given sequence of outer eigensteps even exists, let alone how one should 
find them all. Fortunately, these issues can be addressed.- The key idea is to transition from the eigenvalues 
{An;m}m=i of ^^6 M X M frame operator F^^F* to the eigenvalues {Xn;m}m=i of the nx n Gram matrix F*F,^. 
These two spectra are zero-padded versions of each other. Under this transition, the notion of a sequence of 
outer eigensteps (Definition 2) transforms into an alternative but equivalent notion of inner eigensteps: 

Definition 4. Let {Xn}n=i a-nd {/Un}^=i be nonnegativc nonincreasing sequences. A corresponding sequence of 
inner eigensteps is a sequence of sequences {{A„:m}m=i}ri'=i which satisfies the following three properties: 

(i) ^N-m = Am for every m = l,...,N, 
(ii) {A„_i;„J","l\ C {Xn:r,i}m=i ^r cvcry n = 2, . . . , iV, 
(iii) Em=i ^n-m = Em=i /^™ ^^r every n = l,...,N. 

Note the terms "outer" and "inner" follow from the matrices that they arc derived from: outer eigensteps arise 
as spectra of frame operators, which are sums of outer products; inner eigensteps arise from Gram matrices, which 
arc tables of inner products. In the next result, we formally verify that one may naturally identify a sequence of 
outer eigensteps with a sequence of inner eigensteps, and vice versa, provided one zero-pads appropriately: 

Theorem 5. Given nonnegative nonincreasing sequences {A„}^^2^ and {/x„}^^2j where A„ = for every n > M , 
every choice of inner eigensteps corresponds to a unique choice of outer eigensteps and vice versa, the two being 
zero-padded versions of each other. Specifically, inner eigensteps {{Xn\m}m=i}n=i correspond to outer eigensteps 
{{Xn-m}m=i}n=o> whcrc Xn;ni — wkencver n = or m > n. Conversely, outer eigensteps {{A„;m}m=i}^=o 
correspond to inner eigensteps {{Xn-m}m=i}n=i' where Xn-m = whenever m > M. 

Moreover, for every n = 1, . . . , N, {A„;m}m=i *■' ^^^ spectrum, of the frame operator F^F* of Fn — {fm}m=i 
if and only if {Xn-m}m=i *■' ^^^ spectrum of the Gram matrix F*F,^. 

Proof. First, take outer eigensteps {{Xn-m\m=i}n=o^ ^^d consider {{A„;„i}5^^i}^^i, where we define 

Xn\m = whenever m > M. (4) 

Then Definition 4.i follows from Definition 2.ii when m < Af , and from (4) and the assumption that A„ = for 
every n > M when m> M . Next, we note that Definition 2. iii gives 

Xn;m + l < A„_i;„i < A„:,„ Vto = 1, . . . , A/ — 1, (5) 

< A„_i,M < A„:A/, (6) 

for every n = 1, . . . ^N . To prove Definition 4.ii, pick any n = 2, . . . , N. We need to show A„;m+i < Xn-i;m < 
An;m for every TO = l,...,n — 1. This follows directly from (5) when n < M or when n > M and m < M. If 
n > M and m ~ M, then (4) and (6) together give 

A,i:A/ + l = < Xn-l^M < A„;M- 

Also, (4) gives that Xn-m+i < Xn-i-m < Xn-m bccomcs < < whenever n > M and m > M. For 
Definition 4. iii, note that when n > M, (4) and Definition 2.iv together give 

n M n 

771—1 777 — 1 777—1 

Furthermore, if n < M, then Definition 2.i and Definition 2. iii together give 

Xn;m = whenever m > n, (8) 



and so (8) and Definition 2.iv together give (7). 

Now take inner eigensteps {{^n:m}m=i}n=iJ ^^^^ consider {{Xnim}m=i}n=0 7 where we define 

An;m = whenever m > n. (9) 

Then Definition 2.i follows directly from (9) by taking n = 0. Also, Definition 2.ii follows from Definition 4.i 
since M < N. Next, Definition 4.ii gives 

A„:m+i < A„_i;to < A„;to Vn = 2,..., A^, m = l,...,n - 1. (10) 

Using the nonnegativity of {Xn}n=i along with Definition 4.i and an iterative application of the left-hand in- 
equality of (10) then gives 

< Aat = Xn;n < ■■ ■ < A„;„ Vn = 1, . . . , iV. 

Combining this with an iterative application of the right-hand inequality of (10) then gives 

< A„i;„i < ■ • ■ < Xn:m Vn > TO. (11) 

For Definition 2.iii, we need to show (5) and (6) for every n = 1, . . . ,N. Considering (9), when n = 1, (5) and 
(6) together become Ai;i > 0, which follows from (11). Also, when n > M, (10) immediately gives (5), while (6) 
follows from both (11) and (10): 

< Xn-M+l < Xn-l-^M < Xn-M- 

For the case 2 < n < M, note that (10) gives the inequalities in (5) whenever m < n — 1. Furthermore when 
m = n, (9) gives A„:„+i — A„_i;„ = 0, and so the inequalities in (5) become A„;„ > 0, which follows from (11). 
Otherwise when to > n, the inequalities in (5) become < < by (9). To finish the case 2 < n < M, we need 
to prove (6). When n — M, (6) becomes A„;„ > 0, which follows from (11). Otherwise when n < M, (6) becomes 
< < by (9). We now have only to prove Definition 2.iv. For n < M, (9) and Definition 4.iii together imply 

AI n n 

rn — 1 rrt—1 m—1 

Next, note that (11), Definition 4.i, and our assumption that A„ = for every n > M gives 

< Xn-rn < Xj^-rn ~ Xm ~ whcncvcr n > m > M. 
Thus. A„;,„ = whenever n > m > A/; when n > Af , we can combine this with Definition 4.iii to get (12). D 

This result, when coupled with a complete constructive characterization of all valid outer eigensteps — itself 
a nontrivial result — provides a systematic method for constructing any and all valid inner eigensteps,^ thereby 
making Step A of Theorem 3 explicit. 

4. CONSTRUCTING FRAMES FROM EIGENSTEPS 

Our second issue with the algorithm of Theorem 3 is that Step B apparently requires a large amount of tedious 
linear algebra. To be precise, in order to implement Step B, one appears forced to compute the eigenvectors of 
F^F* for each n = 1, . . . , A^— 1. Though this is not difficult — by construction, the eigenvalues of F^F* are known 
to be {Xn;m}m=i — ^his does not lend itself to elegant closed-form expressions for the frame vectors themselves. 
Fortunately, this process can be made surprisingly explicit: see Theorem 7 of Cahill et al's work.'^ We do not 
present these details here. However, for the sake of the interested reader, we do provide the following MATLAB 
code that implements this improved version of Step B. 

Here, for the sake of simplicity, 1^ = I for all n. The following two functions must be placed in the same 
directory in order to execute the code. The first function constructU.m implements Step B. Here the output of 
the function is a slightly modified version of Steps B.4 and B.5. The function constructU.m returns f/^/„+i and 
UnU„_^_i. We recursively call constructU.m and then multiply the ouputs at each iteration in order to calculate 
/„ and [/„ for n ~ 2, . . . , N. This step is accomplished by the function constructFrame.m which outputs the 
final sequence of vectors F — {fn}n=i whose frame operator FF* has spectrum {Xm}m=i and which satisfies 
ll/nP = /in for all n. 



function [U, Uf] = constructU(El , E2) 

% Description: This function implements Steps B.1-5 of the algorithm to 

7, explicitly construct any and all sequences of vectors whose 

°/o partial-f rsime operator spectra match the eigensteps chosen 

°/, in Step A. Here, we assume VI,..., Vn are the identity. 

7. Call: [U, Uf] = constructU(El, E2) 

'It El = Spectrum at (n) 

7, E2 = Spectrum at (n+1) 

7. Output: U = U_(n)* U_(n+1) 

7. Uf = U_(n)* f_(n+l) 

% File: constructU.m 

7oSpectra must be row vectors and listed in descending order***************** 
if ~isrow(El), E1=E1'; end 
if ~isrow(E2), E2=E2'; end 
El = sort(El, 'descend'); 
E2 = sort(E2, 'descend'); 

M = length(El); 

7oFind indices of unique elements (Step B .2)******************************** 
Rl = El; 7oUnique set of eigenvalues of El 
R2 = E2; 7oUnique set of eigenvalues of E2 
for i = 1:M 

[tf, loc] = ismember(El(i) , R2) ; 
if tf == 1 

[tf ,loclast] =ismember (El (i) , Rl) ; 
Rl(loclast) = -1; 
R2(loc)=-l; 
end 
end 

7Blndex sets of unique elements of El ajid E2, respectively. 
1 = find(Rl >=0); 
J = find(R2 >=0) ; 

Ml = length (I); 
M2 = M-Ml; 

Rl = Rl(l); 
R2 = R2(J); 

"/Construct column and row vectors (Step B. 3)******************************* 
for i = 1:M1 

P(i) = sqrt(-prod(Rl(i)-R2)/prod(Rl(i)-Rl(find(Rl~=Rl(i))))); 

Q(i) = sqrt(prod(R2(i)-Rl)/prod(R2(i)-R2(find(R2~=R2(i))))); 
end 

7oConstruct difference matrix 
for i = 1:M1 

D(i,:) = l./(R2-Rl(i)); 
end 



W =(P'*Q) .*D; 

y.Create Block Diagonal Matrix 
UU=blkdiag(W,eye(M2)); 

ZPermute Row and Columns; 
PRow = permMatd.M) ; 
PCol = permMat(J,M) ; 

"/oCompute U and Uf (Modified Steps B.4 and B. 5) ***************************** 
U = PRow*UU*inv(PCol) ; 
Uf=zeros(M,l); 
Uf(I) = P; 

7oDef ine permutation matrix given the unique index set 1 

function P = permMat(I,M) 

m = 1:M; 

pi = [I setdif f (m, 1)] ; 

Id = eye(M) ; 

P = Id(l:M,pi); 

function tf = isrow(E) 
[rows , cols] =size (E) ; 
if rows==l 

tf=l; 
else 

tf=0; 
end 

function F = constructFrEime(E,Ul) 

7. Description: This function implements Step B of the algorithm to 

'/o explicitly construct any and all sequences of vectors whose 

% partial-frame operator spectra match the eigensteps chosen 

'/, in Step A. Here, we assume VI,..., Vn are the identity. 

7o Call: F = constructFrameCE, Ul) 

7o E = Matrix of eigensteps 

°/t Ul = Initial unitary matrix 

7o Output: The frame, F. 

7« File: constructFrame .m 

UU(:,:,1) = Ul; 
U(:,:,l) = UU(:,:,1); 
F(:,l) = U(:,l,l); 
[M,N] = size(E); 

for i = 2:N 

[UU(:,:,i), Uf(:,i)] = constructU(E( : , i-1) ,E( : , i) ) ; 
U(:,:,i) = eye(M); 

7.Multiply matrices to find new U 

for j = l:i, U(:,:,i) = U( : , : , i)*UU( : , : , j) ; end 



"/oMultiply to find new f 
Temp = eye(M) ; 

for j = l:i-l; Temp = Temp*UU( : , : , j) ; end 
F(:,i) = Temp*Uf (: ,i); 
end 

The following example reproduces the results of Example 8 of Cahill et al's work.^ 

» E = [0 2/3 5/3; 1/3 4/3 5/3 5/3; 1 5/3 5/3 5/3 5/3] 

E = 







1.0000 




0.3333 
1.6667 



1.3333 
1.6667 


0.6667 
1.6667 
1.6667 


1.6667 
1.6667 
1.6667 


» Ul=eye(3) 










Ul = 










1 
1 





1 








» F = constructFrame(E,Ul) 






F = 










1.0000 




0.6667 

0.7454 




-0.4082 

0.9129 




-0.1667 
0.3727 
0.9129 


0.1667 

-0.3727 

0.9129 
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